fix(streaming): keep the served service_tier on streamed chunks and spend rows by devin-ai-integration[bot] · Pull Request #42870 · BerriAI/litellm

devin-ai-integration · 2026-09-24T03:42:31Z

TLDR

Problem this solves:

Streamed OpenAI calls lost the service_tier OpenAI stamped on the response
Spend rows for streamed calls recorded no served tier
Some relayed chunks (all of them on the Responses bridge) dropped service_tier

How it solves it:

Stream reassembly keeps the last tier the provider stamped on a chunk
The Responses-to-chat bridge remembers the tier from response.created and stamps it on every translated chunk
Final and usage-only chunks carry the tier through to the caller and spend logging
The proxy fast serializer includes service_tier on every chunk
The /v1/messages chat-adapter stream (and the router's fallback wrapper and the response-cache writer around it) exposes the chat chunks it wraps, so a client disconnect mid-stream bills the partial spend and its tier the way /v1/chat/completions already does
The generic OpenAI-compatible SSE chunk parser (hosted_vllm, openai_like and every provider inheriting OpenAIChatCompletionStreamingHandler) keeps service_tier on the parsed chunk instead of dropping it, so those deployments can bill *_priority rates too
Databricks streams keep the served tier on parsed chunks and the Databricks cost path bills it (LIT-8121)
Cost calc now bills the tier the provider served over the tier the caller requested: a served flex/balanced/priority/fast/ultrafast wins, a served default/standard (or Gemini ON_DEMAND) forces base rates, and auto/scale/unknown/absent echoes fall back to the requested tier

Intentional product change: a request for priority that the provider downgrades to default is now billed at base rates instead of priority rates, matching the provider's own invoice

User Flow

Before: a developer streams a chat completion and can not tell from the logs which tier OpenAI actually served, so tier pricing is unverifiable

They send POST http://localhost:4000/v1/chat/completions with "stream": true, stream_options.include_usage and no service_tier
OpenAI serves the call at "service_tier": "default", but the final usage chunk comes back without service_tier (on gpt-5.x models via the Responses bridge, no chunk has it)
They GET http://localhost:4000/spend/logs?request_id=... and the row's metadata.cost_breakdown.service_tier is null

After: the same request shows the served tier on every chunk and on the spend row

They send the same POST http://localhost:4000/v1/chat/completions with "stream": true, stream_options.include_usage and no service_tier
Every chunk, including the usage chunk, comes back with "service_tier": "default"
GET http://localhost:4000/spend/logs?request_id=... shows metadata.cost_breakdown.service_tier as "default" and the row is billed at that tier's rates

Second flow, the caller asks for a tier and the provider serves a different one

Before: they send the same POST with "service_tier": "priority", OpenAI is short on priority capacity and serves it at "service_tier": "default" (visible on every chunk), and the spend row bills priority rates anyway because cost calc read the requested tier first

After: the same request bills base rates and the spend row shows "default", matching the provider invoice. A request for priority that comes back priority, auto or scale, or with no tier at all, still bills priority, so callers on providers that do not echo the tier see no change

Integration coverage across the related tickets

tests/integration/spend/test_service_tier_stream_billing.py runs a real proxy, Postgres and a scripted upstream that stamps the served tier on the stream. Each test bills at distinct default and tiered rates, so a wrong tier cannot produce the expected spend. Run on origin/main (b248b1c) with the same test file, then on this branch:

test	ticket	main	this PR
completed chat stream	LIT-8514	fail (chunks carry no tier)	pass
disconnected chat stream	LIT-8514	fail (billed 0.016 at default, expected 0.16)	pass
completed /v1/messages stream	LIT-8514	fail (billed 0.11, expected 1.1)	pass
disconnected /v1/messages stream	LIT-8514	fail (no spend row written)	pass
azure chat stream	LIT-2850	fail (chunks carry no tier)	pass
databricks chat stream	LIT-8121	fail (chunks carry no tier)	pass
responses bridge stream	LIT-8514	fail (first chunks carry no tier)	pass
gemini flex via trafficType	LIT-6287, LIT-6292	pass	pass
requested priority, served default	served-over-requested	fail (billed priority)	pass
requested priority, served auto	served-over-requested	pass	pass

Mutation: revert _get_service_tier_from_chunks to return None and every priority row above bills at default rates; drop the service_tier kwarg from DatabricksChatResponseIterator.chunk_parser and only the databricks row goes red; make _resolve_billable_service_tier return the requested tier first and the downgrade row bills priority again

Linear ticket

Resolves LIT-8514

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Setup: proxy on localhost:4000 with a Postgres spend log DB and a deployment nano-tier registered as openai/gpt-4.1-nano (real OpenAI key). Same request both times:

curl -sN http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model":"nano-tier","messages":[{"role":"user","content":"Say hi"}],"stream":true,"stream_options":{"include_usage":true}}' > stream.txt
echo "chunks: $(grep -c '^data: {' stream.txt)  with service_tier: $(grep -c '"service_tier":"default"' stream.txt)"
RID=$(grep -o 'chatcmpl-[A-Za-z0-9]*' stream.txt | head -1); sleep 15
curl -s "http://localhost:4000/spend/logs?request_id=$RID" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | python3 -c "
import json,sys; r=json.load(sys.stdin)[0]; cb=r['metadata']['cost_breakdown']
print(json.dumps({'request_id':r['request_id'],'model':r['model'],'spend':r['spend'],'service_tier':cb['service_tier'],'input_cost':cb['input_cost'],'output_cost':cb['output_cost']},indent=1))"

Before (`a3d791f`)

Streamed chunks carry the served tier

Ran the curl above

Observed:

chunks: 5  with service_tier: 4

The final usage chunk has no service_tier:

data: {"id":"chatcmpl-ERUcP5GEoz6ZvN0Eh7wxoej1n0HxV","created":1790221261,"model":"nano-tier","object":"chat.completion.chunk","system_fingerprint":"fp_a8d11c9b56","choices":[{"index":0,"delta":{}}],"usage":{"completion_tokens":2,"prompt_tokens":9,"total_tokens":11,...,"cost":1.6999999999999998e-6}}

Spend row records the served tier

Queried /spend/logs for that request id

Observed:

{
 "request_id": "chatcmpl-ERUcP5GEoz6ZvN0Eh7wxoej1n0HxV",
 "model": "openai/gpt-4.1-nano",
 "spend": 1.7e-06,
 "service_tier": null,
 "input_cost": 9e-07,
 "output_cost": 8e-07
}

After (`e6722a9`)

Streamed chunks carry the served tier

Ran the curl above

Observed:

chunks: 5  with service_tier: 5

The final usage chunk now carries it:

data: {"id":"chatcmpl-ERUarhGzcTOQV7GhLS6wZDLTnkUUV","created":1790221165,"model":"nano-tier","object":"chat.completion.chunk","system_fingerprint":"fp_a8d11c9b56","choices":[{"index":0,"delta":{}}],"usage":{"completion_tokens":2,"prompt_tokens":9,"total_tokens":11,...,"cost":1.6999999999999998e-6},"service_tier":"default"}

Spend row records the served tier

Queried /spend/logs for that request id

Observed:

{
 "request_id": "chatcmpl-ERUarhGzcTOQV7GhLS6wZDLTnkUUV",
 "model": "openai/gpt-4.1-nano",
 "spend": 1.7e-06,
 "service_tier": "default",
 "input_cost": 9e-07,
 "output_cost": 8e-07
}

Responses bridge (gpt-5-nano, deployment `nano5-tier`), same curl

Before (a3d791f):

chunks: 13  with service_tier: 0
{"request_id": "chatcmpl-ERV6rbGcRRj72um9gYk7Her31VoR2", "model": "openai/gpt-5-nano", "spend": 8.48e-05, "service_tier": null, ...}

After (69fe084):

chunks: 13  with service_tier: 13
{"request_id": "chatcmpl-ERV4d5Mj7NGGMbp2nQYWGxQSJygFH", "model": "openai/gpt-5-nano", "spend": 5.96e-05, "service_tier": "default", ...}

The last two frames on the tip:

data: {"id":"chatcmpl-ERV4d5Mj7NGGMbp2nQYWGxQSJygFH","object":"chat.completion.chunk","created":1790223012,"model":"nano5-tier","service_tier":"default","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"chatcmpl-ERV4d5Mj7NGGMbp2nQYWGxQSJygFH",...,"choices":[{"index":0,"delta":{}}],"usage":{"completion_tokens":148,"prompt_tokens":8,"total_tokens":156,...},"service_tier":"default"}

/v1/messages disconnect (gpt-4.1-nano, chat-adapter route)

Recipe on both legs: register openai/gpt-4.1-nano via POST /model/new with input_cost_per_token 4e-5, output_cost_per_token 8e-5 and *_priority 6e-5 / 1.6e-4, then abandon a stream and read the key's spend rows:

timeout 3s curl -sN -X POST http://localhost:4000/v1/messages \
  -H "Authorization: Bearer $VK" -H "Content-Type: application/json" \
  -d '{"model":"tier-discount-chat","max_tokens":400,"stream":true,
       "messages":[{"role":"user","content":"Tell me a very long detailed story about a dragon and a lighthouse keeper, at least 500 words"}]}'
sleep 90
curl -s "http://localhost:4000/spend/logs?api_key=$VK" -H "Authorization: Bearer $MK"

Before (69fe084): the client saw message_start, content_block_start and text deltas, the proxy logged Recorded streaming client disconnect with error_code=499, and /spend/logs returned [] after the flush (writer logged Spend Logs transactions: 0)

After (047b506): same stream, proxy logged Billing partial streamed spend for 3 chunks after client disconnect, and the row landed:

{
    "request_id": "msg_b9dda6dd-6415-4413-b199-b15e81de63be",
    "call_type": "anthropic_messages",
    "spend": 0.00124,
    "prompt_tokens": 27,
    "completion_tokens": 2,
    "model": "openai/gpt-4.1-nano",
    "metadata": {"cost_breakdown": {"input_cost": 0.00108, "output_cost": 0.00016, "total_cost": 0.00124, "service_tier": "default"}}
}

The two new e2e tests (-k "responses_stream or messages_stream") pass against that proxy: 2 passed in 120s, live OpenAI

Validation on this branch: tests/e2e/quota_management/spend_tracking/test_service_tier_pricing_e2e.py streamed cases failed on the merge base and pass on the tip (2 passed, 119s, live OpenAI). Unit regression tests in test_streaming_chunk_builder_utils.py, test_streaming_handler.py, test_streaming_helpers.py and the Responses transformation test file. Mutations reverting the chunk builder tier scan, the reverse scan order, the Responses terminal tier copy, the Responses stream-scoped tier memo (test_every_bridged_chunk_after_response_created_carries_the_served_service_tier), the fast serializer field and the final response stamping each fail at least one of those tests. python -m coverage_registry.collector --strict passes with the two new registry cells. scripts/type_discipline_gate.py, scripts/ruff_strict_gate.py, scripts/type_check_gate.py --base origin/main and the test-quality gate all pass

Integration matrix (scripted upstream, real proxy, Postgres and Redis cache on)

tests/integration/spend/test_service_tier_stream_billing.py: the upstream stamps service_tier: "priority" on every chunk and the deployment registers 10x priority rates, so a bill at the wrong tier can not match. 4 passed in 19.67s on the tip

test	spend	tokens	cost_breakdown.service_tier
completed /v1/chat/completions	1.1	30/40	priority
disconnected /v1/chat/completions	0.16	14/1	priority
completed /v1/messages (chat adapter)	1.1	30/40	priority
disconnected /v1/messages (chat adapter)	0.18	16/1	priority

Before the last two commits the completed /v1/messages row billed 0.11 at default rates with a null tier (the OpenAI-compatible chunk parser dropped service_tier), and the disconnected one wrote no row at all (with litellm.cache on, AnthropicMessagesStreamCacheWriter sat between the router wrapper and the chat stream and hid .chunks)

Endpoint x outcome coverage

	complete stream	client disconnect mid-stream
/v1/chat/completions	billed at served tier (e2e `test_streamed_call_records_and_bills_the_served_tier`)	partial spend billed with the tier (existing `TestStreamingClientDisconnectBilling` suite, chunk builder tier tests)
/v1/chat/completions via the Responses bridge	billed at served tier (bridge unit test, live proof above)	same partial-spend path as chat, chunks stamped by the bridge
/v1/responses (native)	already worked: tier is read off `response.completed` (unit `test_responses_completed_event_bills_the_served_service_tier`, e2e `test_responses_stream_records_the_served_tier`)	not billed at all, tracked separately in LIT-8603 (the Responses iterator keeps no events and usage only arrives on `response.completed`)
/v1/messages (OpenAI backend)	billed at served tier by the inner stream (e2e `test_messages_stream_records_the_served_tier`); the Anthropic wire format has no tier field	chat-adapter path (`LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES`, or any non OpenAI/Azure backend) fixed here: partial spend billed with the tier (unit `test_disconnect_bills_partial_spend_for_anthropic_adapter_stream`, fails before the fix, integration matrix above, live proof below). The default OpenAI route goes through the Responses adapter and bills nothing on disconnect, tracked with the native Responses gap in LIT-8603

Type

🐛 Bug Fix

Caveats (if any)

Low

Partial billing on disconnect for /v1/messages only reaches the chat-adapter route. The default OpenAI route uses the Responses adapter, whose iterator keeps no events and gets usage only on response.completed, so it still bills nothing on disconnect (LIT-8603)
Groq rewrites a missing echo to auto and OpenAI can echo scale; both count as no signal, so the requested tier still decides there
An explicit service_tier argument passed straight into completion_cost (the Responses WS partitioner) still overrides both request and response
The integration matrix uses hosted_vllm/ deployments for /v1/messages because OpenAI and Azure backends take the Responses adapter by default; the OpenAI chat-adapter route is covered by the unit test and the live proof
The usage-only unit fixture stamps the tier on the source chunk, so the usage-chunk copy path is covered by the live e2e test rather than the unit test

QA runbook

tests/e2e/quota_management/spend_tracking/test_service_tier_pricing_e2e.py::TestServiceTierPricing::test_streamed_call_records_and_bills_the_served_tier - a streamed call with no tier requested records the tier OpenAI served on its spend row and bills at that tier's rates
- POST /model/new with the master key registering openai/gpt-4.1-nano with distinct input_cost_per_token, output_cost_per_token, input_cost_per_token_priority and output_cost_per_token_priority (needs OPENAI_API_KEY and STORE_MODEL_IN_DB=True)
- POST /v1/chat/completions to that model with "stream": true, stream_options.include_usage and no service_tier; note the service_tier on the chunks
- GET /spend/logs?request_id= and expect metadata.cost_breakdown.service_tier to equal the tier seen on the chunks and spend to equal prompt and completion tokens times that tier's rates
- Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
tests/e2e/quota_management/spend_tracking/test_service_tier_pricing_e2e.py::TestServiceTierPricing::test_every_streamed_chunk_carries_the_served_tier - every relayed chunk of a streamed OpenAI call carries the served service_tier
- POST /v1/chat/completions to the same model with "stream": true and stream_options.include_usage
- Expect every data: chunk, including the final usage-only chunk, to contain the same non-empty service_tier
- Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
tests/e2e/quota_management/spend_tracking/test_service_tier_pricing_e2e.py::TestServiceTierPricing::test_responses_stream_records_the_served_tier - a streamed /v1/responses call records the tier carried on response.completed on its spend row
- POST /v1/responses to the same model with "stream": true; note response.service_tier on the response.completed event
- GET /spend/logs?request_id= and expect metadata.cost_breakdown.service_tier to equal that tier
- Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
tests/e2e/quota_management/spend_tracking/test_service_tier_pricing_e2e.py::TestServiceTierPricing::test_messages_stream_records_the_served_tier - a streamed /v1/messages call on an OpenAI-backed deployment records the served tier on its spend row
- POST /v1/messages to the same model with "stream": true and a fresh key
- GET /spend/logs?api_key= and expect the row's metadata.cost_breakdown.service_tier to be a non-null tier that has custom rates registered
- Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/53a7da17cdb047a196848100235a5421
Open in Devin Desktop: https://app.devin.ai/desktop/session/53a7da17cdb047a196848100235a5421?variant=devin
Requested by: @kerry-berri

…hunks and spend rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration · 2026-09-24T03:42:33Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

mateo-berri · 2026-09-24T03:43:18Z

@greptileai

mateo-berri · 2026-09-24T03:43:19Z

bugbot run

greptile-apps · 2026-09-24T03:46:12Z

Confidence Score: 5/5

[Medium risk] Adds service tier tracking to streaming responses.

The PR appears safe to merge; no outstanding or new actionable findings remain.

Summary

The PR carries provider-served tiers through streamed chat chunks, stream reassembly, and spend logging, and prices calls using the served tier when available.

Exposes the inner chat stream to /v1/messages disconnect billing through its adapter, cache, and router wrappers.
Adds integration and unit coverage for served-tier billing across chat, Messages, Responses bridging, Azure, Databricks, and Gemini streams.

_{Reviews (13) · Last reviewed commit: "refactor(cost): drop explanatory comment..."}

cursor

Stale Bugbot comment from a previous run.

codspeed · 2026-09-24T03:51:16Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_stream_served_service_tier (aaf6c94) with main (d2cbc94)}

codecov · 2026-09-24T03:52:35Z

Codecov Report

❌ Patch coverage is 97.41379% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/router.py	84.61%	2 Missing ⚠️
litellm/proxy/proxy_server.py	50.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ge chunk Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

CLAassistant · 2026-09-24T04:09:20Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

❌ devin-ai-integration[bot]
❌ kerry-berri
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

mateo-berri · 2026-09-24T04:09:24Z

@greptileai

mateo-berri · 2026-09-24T04:09:25Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…rtial spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

… paths Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

litellm-stampbot · 2026-09-25T00:44:09Z

@greptileai

litellm-stampbot · 2026-09-25T00:44:10Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…s bill partial spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

…tream wrapper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

…_service_tier Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> # Conflicts: # tests/test_litellm/litellm_core_utils/test_litellm_logging.py

litellm-stampbot · 2026-09-27T00:52:57Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…wargs dict Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

litellm-stampbot · 2026-09-27T01:55:24Z

@greptileai

litellm-stampbot · 2026-09-27T01:55:25Z

bugbot run

cursor · 2026-09-27T01:55:28Z

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

litellm-stampbot · 2026-09-29T17:13:54Z

@greptileai

litellm-stampbot · 2026-09-29T17:13:54Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit aaf6c94. Configure here.}

mateo-berri

LGTM. Thanks!

mateo-berri · 2026-09-29T20:16:26Z

but the final usage chunk comes back without service_tier (on gpt-5.x models via the Responses bridge, no chunk has it)

Follow-up check on whether this is OpenAI not sending us this stuff, or it's just that we were getting them from the provider's chunks, but just not forwarding it to the client or using it would be good

@devin-ai-integration can you do the follow-up and post a GitHub comment on what's happening?

devin-ai-integration · 2026-09-29T20:20:43Z

OpenAI does send it: the Responses stream carries service_tier on response.created/in_progress/completed (full Response object), never on delta events. Before this PR the bridge only stamped the terminal chunk and reassembly dropped it, so the flow wording "no chunk has it" overstated it

Resolves the conflict at the end of tests/unit/litellm_core_utils/test_streaming_handler.py, where BerriAI#42870 appended a test after the same context: both blocks kept, main's first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…served at default #42870 added both the rule that a served default or standard tier bills at base pricing and records no service_tier, and streamed tests expecting the row to record 'default'. They have failed on every scheduled litellm-e2e run since. The tests now map the served tier to the pricing basis the bill must record and check input is billed at that basis's rate; the messages case registers custom rates so the rate check has something to compare against

* test(ci): add used_client_oauth_token to the GCS pub/sub spend-log golden #43063 stamps used_client_oauth_token into spend-log metadata, so test_async_gcs_pub_sub_v1 failed on main with an extra metadata key * test(ui): give the auto-router threshold save wait room for the availability debounce #42625 keeps Save disabled while a 300ms-debounced availability check runs. This test waits for Save right after the change, so the whole debounce lands inside waitFor's 1s default and it times out under CI load. It is the recurring UI Unit Tests failure on main since #42625 landed * test(e2e): expect no pricing tier on bills for streamed calls OpenAI served at default #42870 added both the rule that a served default or standard tier bills at base pricing and records no service_tier, and streamed tests expecting the row to record 'default'. They have failed on every scheduled litellm-e2e run since. The tests now map the served tier to the pricing basis the bill must record and check input is billed at that basis's rate; the messages case registers custom rates so the rate check has something to compare against * test(e2e-ui): wait for the call-id search before hovering the logs row The row the spec hovers is already on the unfiltered first page, so it was found before the search request returned. The search response then re-rendered the table under the mouse, and the Base UI tooltip never opened. Reproduced with Playwright against a local proxy: hovering right after the fill never shows the tooltip, hovering after the search response shows the call id every time * test(e2e): run the Together structured-output case on the hybrid Qwen with reasoning off The case picked the cheapest Together row flagged supports_response_schema. DeepSeek-V4-Flash-0731 hit its cost-map deprecation date on 2026-09-29, so the pick moved to GLM-5.3-Flash, a reasoning-only model that spends the 1024-token budget thinking and returns content=None. Qwen3.5-9B is the pinned hybrid model the reasoning_effort=none case already exercises, and Together lists it with structured output support * test(integration): read the agent 365 guardrail status by its own name in spend logs The MCP shard runs under xdist against one database, and a sibling file creates a default_on pre_mcp_call content filter there. The owned proxy reloads DB guardrails, so that filter's 'success' entry could land first in guardrail_information and the test read it instead of the agent 365 verdict * test(unit): ignore asyncio's leaked-task records in the budget limiter push-failure log check gc.collect() inside the caplog window can collect a pending task an earlier test left on a closed loop, and asyncio logs 'Task was destroyed but it is pending' into this test's records. The check still counts every LiteLLM logger, and unretrieved task exceptions on this loop still go through the asserted exception handler * test(e2e-ui): fill the create-tag fields inside the dialog #42949 added 'Filter by tag name' and 'Filter by description' inputs to the Tag Management page, so page-wide getByLabel('Tag Name') and getByLabel('Description') match two elements and Playwright's strict mode fails the create step * test(integration): run integration proxies with the CI license Multi-worker proxies start each uvicorn worker in a fresh process, so every worker reads the license from its environment. Forward LITELLM_LICENSE into the proxy and test runner environments * ci: save GitHub Actions caches only from main and bump codecov-action to 5.5.5 Every pull request saved its own uv, maturin, Rust and Prisma caches, about 4.5 GB per PR, so the repository's 10 GB cache budget evicted main's entries within minutes. Pull request jobs then missed every cache, downloaded all dependencies from PyPI and hit the install step timeouts. Pull requests now restore only, and main keeps the caches warm for them. test-linting and check-ui-api-types run only on pull requests and keep saving codecov-action 5.5.4 imports its signing key from the deleted codecovsecurity keybase account, so every upload failed signature verification. 5.5.5 reads it from codecovsecops; the key ID matches the one signing the current CLI * test(unit): join the session-minting thread before collecting the handler asyncio.to_thread resumes the test as soon as the worker sets its result, while the pool thread can still hold the work item and through it the handler. gc.collect() then cannot finalize the handler and the session stays open. A pool that shuts down before the test continues drops that reference * test(integration): relaunch owned proxies that lose their port, expire idle gateway connections early owned_proxy_process released its reserved port and the proxy bound it only after full startup, so another xdist worker or an outgoing connection could take it first and the proxy exited with 'address already in use'. The launch now retries on a fresh port when that happens and stops every failed attempt. uvicorn closes idle keep-alive connections after 5 seconds and httpx expired them at the same 5 seconds, so a request sent right at that mark could reuse a socket the server was closing and get 'Connection reset by peer'. Gateway clients now drop idle connections after 2 seconds * ci(circleci): give the base SDK wheel build the same 30 minute no-output window as the Windows build The release profile builds with fat LTO and one codegen unit, so the final link of litellm-cache-s3 runs silently for minutes. Successful builds take 711 to 749 seconds, right at the default 10 minute no-output limit, and about 30% of recent runs were killed there * test(integration): model the budget-reset database outage as 10 seconds instead of 5 refused connections The proxy retries the database about every 30 seconds and each retry opens roughly one connection, so a 5-connection outage took 3 to 4 retries to clear and recovery landed between 60 and 90 seconds, straddling the test's 80 second reset window. A fixed 10 second outage still refuses the immediate reconnect and recovers on the next retry * ci: move the unit-test uv cache split into a composite action check_workflow_startup_safety sums every setup step's timeout, so the save and restore variants each counted 5 minutes although only one runs. One composite step keeps the setup ceiling at 35 minutes * test(unit): point tiktoken at the bundled cache for every unit test The rust_bridge tokenizer tests loaded o200k_base before any test in their xdist worker had imported default_encoding, so tiktoken fell back to the temp cache and tried to download under pytest-socket. Move the session fixture from litellm_core_utils/conftest.py to the root unit conftest. * test(integration): answer model discovery probes in the hosted_vllm wire tests The router's periodic upstream model info refresh sends GET /v1/models to hosted_vllm deployments, so a wire server that is live during a refresh sees an extra request. Answer the probe with an empty model list and leave it out of the provider-call assertions, matching the responses bridge tests.

* chore(codeowners): drop UI, migration, and CODEOWNERS self owners (#43653) * chore(codeowners): drop UI and migration code owners Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(codeowners): drop CODEOWNERS self-owner line Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers (#43641) * feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fireworks_ai): drive the router request test through an httpx MockTransport Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(fireworks_ai): let custom firerouter/<models> IDs inherit the firerouter row's capabilities Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fireworks_ai): integration coverage for router short names forwarding tool_choice and reasoning_effort Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fireworks_ai): assert tool definitions reach the router upstream Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat: add model leaderboard page (#43649) * feat(proxy): add model leaderboard analytics * feat: add model insights task and range constants * feat: record task type from task tags in model usage rollup * feat: serve 365 days of model insights by UTC date * test: cover task tag resolution in model usage rollup * test: update model insights range limit test to 365 days * chore: regenerate dashboard api types for model insights * feat: add model insights aggregation helpers * test: cover model insights aggregation helpers * feat: redesign model leaderboard with stacked bars, treemap and ranking * test: update model leaderboard view test * feat: mark model leaderboard as beta in sidebar * chore: sync schema.prisma copies from root * fix: only treat task: prefixed tags as model insight tasks * feat: add metric type for model insights ranking * fix: rank model insights by selected metric and scope detail queries to ranked deployments * test: plain tags are not model insight tasks * test: cover metric ranking, deployment scoping and rollup round trip * fix: build model insights weeks and halves from the requested date range * test: cover empty weeks and range-based change comparison * fix: refetch by metric, show load errors and ignore stale responses * test: cover metric refetch and error state * feat: define model insight tasks in a JSON file * feat: return task labels and categories from model insights * feat: load model insight tasks from JSON * refactor: validate rollup task tags against the JSON task list * feat: serve the task list with model insights * refactor: drop hardcoded task list from constants * build: ship model insight tasks JSON in the wheel * test: cover model insight task JSON * refactor: take task labels and categories from the API * test: pass task info to task tile builder * refactor: color treemap by API-provided category * test: include tasks in model leaderboard fixture * fix: make daily model usage migration idempotent * feat: bound the model insights task query size * fix: compute task breakdown independent of the chart metric * test: task breakdown is stable across chart metrics * chore: regenerate lazy openapi snapshot for model insights * chore: regenerate dashboard api types for model insights * fix: keep previous ranking dimmed while a new metric loads * test: cover stale metric state in model leaderboard * refactor: drop task row cap constant * fix: return the full task breakdown instead of a truncated one * test: task query is not truncated * feat: add task summary types for model insights * feat: summarise tasks server-side on a separate model insights endpoint * test: cover the model insights tasks endpoint * chore: regenerate lazy openapi snapshot for model insights tasks * chore: regenerate dashboard api types for model insights tasks * refactor: drop client-side task aggregation * test: remove client-side task aggregation tests * feat: load task breakdown separately from the chart metric * test: task breakdown is not refetched on chart metric change --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix(google_genai): preserve proxy_server_request in completion adapter (#43536) * fix(otel): send cache and reasoning tokens in langfuse usage_details (#43553) * fix(otel): send cache and reasoning tokens in langfuse usage_details The OTel V2 Langfuse mapper only sent input, output and total, so cache reads, cache writes and reasoning tokens never reached Langfuse. Emit them as input_cached_tokens, input_cache_creation and output_reasoning_tokens, and send input/output net of those buckets so Langfuse does not price the same tokens twice. Fixes #43542 * fix(otel): drop redundant comments from the usage_details change * fix(vertex_ai): forward the per-turn-control beta for per-message output_config (#43558) Claude Code attaches output_config to mid-conversation system messages and sends the per-turn-control-2026-07-01 beta with it. The Vertex beta map dropped that beta, so Vertex rejected the body with 'messages.N.output_config: Extra inputs are not permitted'. Forward the beta for vertex_ai, the way azure_ai already does, and add it on the Vertex Messages path whenever a message carries output_config. * fix(cost-map): add tool calling and reasoning flags, correct max output for nebius DeepSeek-V4.1-Flash (#43588) * fix(proxy): resolve model_group_alias in the zero-cost budget predicate (#43512) `_is_model_cost_zero()` reads a group's cost through `Router.get_model_group_info()`, which resolves `model_group_alias`, and then gates that on `_is_cost_explicitly_configured()`, which scanned `Router.model_list` for an exact `model_name` match. Alias names live only in `Router.model_group_alias` and are never `model_name` entries, so the scan found nothing and returned False. That False means "the zero cost was defaulted, not configured" (the sparse auto-registration gate added for #24770), so a model priced explicitly at 0 had budget enforced against it when requested through an alias, while the same deployment under its own name was exempt. Both names route to the same deployment and add nothing to spend. The two lookups in one function disagreeing is the bug, so they now share one resolution: `_is_cost_explicitly_configured()` resolves through `Router.get_model_list()`, the same alias-aware path `get_model_group_info()` takes. That also reaches a deployment which prices itself through its `model_info` block, whose cost-map entry lands under the deployment id. `_group_declares_explicit_cost()` was an alias-aware copy of this function, wired only into `model_has_no_cost_mapping()` and never into the budget path; its body is what `_is_cost_explicitly_configured()` now carries, and both callers share it so the two cannot drift apart again. `_has_ptu_flat_cost()` scanned `model_list` the same way and runs after the gate above, so resolving one without the other would let an aliased PTU group — explicit zero per-token price alongside a flat capacity cost — pass as free. It resolves the same way now. Tests cover the predicate and the request path it feeds: over-budget requests through `_should_skip_budget_checks()` into `common_checks()` for an aliased free model (allowed) and an aliased paid model (refused), the predicate for free, paid, PTU, hidden and dangling aliases, and `model_has_no_cost_mapping()` through an alias so the other caller of the shared check stays covered. Unchanged: priced groups (the predicate returns False before the gate), unmapped groups whose zero cost was defaulted (#24770), hidden aliases and aliases pointing at a nonexistent group (`get_model_group_info()` returns None for both, so the cost is unknown and budget is enforced), and non-aliased PTU groups. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: clean up fresh tech debt from 2026-09-28 (#43674) * refactor: clean up fresh tech debt from 2026-09-28 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor: group leaderboard rows in one pass and wrap docstring at 120 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(types): replace Any with proven types in 7 files (#43704) * refactor(types): replace Any with proven types in 11 files Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(types): revert Any changes that broke existing callers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(types): drop prompt factory helper wrappers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover typing sweep surfaces Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): tighten sweep audit tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(security): point readers to the security announcements mailing list signup (#43713) * docs(security): point readers to the security announcements mailing list signup Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(security): formalize the security announcements wording Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(security): tighten the best-effort sentence Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: oliver <oliver@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(rust): add shared llms wire type derives (#43730) * fix(router): stream /v1/messages lifecycle frames live when no fallback can take over (#43600) * fix(router): stream anthropic messages lifecycle frames live when no fallback can take over The /v1/messages streaming wrapper buffered message_start and content_block_start until the first content_block_delta and dropped pings behind buffered frames unconditionally, even for requests no fallback could ever recover. With adaptive thinking on Bedrock or Vertex the client saw no bytes for the whole thinking pass and hit read timeouts. Buffering now applies only while a fallback can still take over (generic or refusal chain resolving), and a ping is always forwarded live since it carries no lifecycle and keeps the connection alive. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): mirror every dispatcher fallback path in the anthropic stream gate Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): skip already-tried order levels in the anthropic stream gate Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): keep a transport-split ping behind buffered lifecycle frames instead of forwarding its head live Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(router): credit the #39566 branch this fix supersedes Co-authored-by: Radu Swigler <radu.porumba@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Radu Swigler <radu.porumba@gmail.com> * feat(cost-map): add baseten DeepSeek-V4.1-Flash-Fast (#43735) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * feat(guardrails): send a configured gateway_name from noma_v2 to Noma (#43678) * feat(guardrails): send a configured gateway_name from noma_v2 to Noma The noma_v2 guardrail accepts a gateway_name param, falling back to the NOMA_GATEWAY_NAME env var. The value is stripped, and when it is non-empty it goes out as a top-level gateway_name field on /litellm/guardrail. The param works for both guardrail: noma_v2 and guardrail: noma with use_v2, and it is appended after the existing constructor params so positional callers keep their meaning * chore(ui): regenerate OpenAPI snapshot and dashboard types for gateway_name The new noma_v2 gateway_name param shows up in the proxy OpenAPI spec, so the lazy snapshot and the generated dashboard types need regenerating * Update litellm/proxy/guardrails/guardrail_hooks/noma/noma_v2.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * chore(cost-map): add openai gpt-6.1-sol from the pricing page (#43738) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): callback credential canary slots C1-C3 and D5 (#43630) * test(integration): credential canary suite harness Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix. * test(integration): widen canary route sweep and harden the rig Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy. * test(integration): descend into any decoded value that can still hold an encoded canary * test(integration): bound canary decoding by depth and decoded bytes * test(integration): scope log-table and spend-log reads to the scenario window * test(integration): sweep spend-log rows in the scenario date window * test(integration): keep spend-log date window summarized * test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot * test(integration): expect 404 from the caller-scoped team membership route * test(integration): use the rig's own master key and expect 404 from submission lookups * test(integration): check the overridden rig key without assuming the default key is unknown * test(integration): callback credential canary slots C1-C3 and D5 Team callback, team callback_settings, config default_team_settings and key metadata.logging Langfuse secrets, a team Datadog dd_api_key, and request-body Langfuse keys (allow_client_side_credentials) must reach only their sink. Each scenario checks its sink received the canary as auth and that the marker is visible at the stored body, the Logs drawer route and the sink. Adds a unit test that the stored request body snapshot carries no callback parameter. * test(integration): give the callback sink waits a wider bound * test(integration): sweep provider requests for callback credentials * test(integration): request-path credential canary slots D1-D4 (#43307) * test(integration): credential canary suite harness Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix. * test(integration): widen canary route sweep and harden the rig Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy. * test(integration): descend into any decoded value that can still hold an encoded canary * test(integration): bound canary decoding by depth and decoded bytes * test(integration): scope log-table and spend-log reads to the scenario window * test(integration): sweep spend-log rows in the scenario date window * test(integration): keep spend-log date window summarized * test(integration): request-path credential canary slots D1-D4 * test(integration): read the Logs drawer and spend-log filter for failed request rows * test(integration): check the marker in each failed row's spend-log filter; run header slots on chat-family routes * test(integration): run D2 on embeddings again; only the client-header slot runs on chat-family routes * test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot * test(integration): pass the slot deployment's model_info id to the route sweep * test(integration): expect 404 from the caller-scoped team membership route * test(integration): use the rig's own master key and expect 404 from submission lookups * test(integration): check the overridden rig key without assuming the default key is unknown * test(integration): sweep proxy logs, metrics, a Datadog intake and the Logs drawer for credential canaries (#43306) * test(integration): credential canary suite harness Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix. * test(integration): widen canary route sweep and harden the rig Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy. * test(integration): descend into any decoded value that can still hold an encoded canary * test(integration): bound canary decoding by depth and decoded bytes * test(integration): scope log-table and spend-log reads to the scenario window * test(integration): sweep spend-log rows in the scenario date window * test(integration): keep spend-log date window summarized * test(integration): sweep proxy logs, metrics, a gzip Datadog intake and the Logs drawer for credential canaries * test(e2e): treat an unset prompt-storage setting as unset and restore it * test(integration): name the Datadog sink slot G1d * test(e2e): search the Logs page for base64 forms of the deployment key * test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot * test(integration): pass the resolved deployment id to the Datadog route sweep * test(integration): expect 404 from the caller-scoped team membership route * test(integration): use the rig's own master key and expect 404 from submission lookups * test(integration): check the overridden rig key without assuming the default key is unknown * chore(cost-map): add azure and openrouter gpt-6.1-sol rows (#43744) Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): classify every credential-bearing param for the canary suite (#43298) * test(proxy): classify every credential-bearing param for the canary suite * test(proxy): classify gcs_path_service_account as secret, run registry in auth-checks shard, check slot ids at import * test(proxy): use one generic slot id for callback and request-body credential params * test(proxy): name a canary slot only for params an integration test plants * test(proxy): classify the SigNoz callback params * test(proxy): move the slot sync note into the module docstring * test(proxy): model unplanted credential params as their own classification * test(security): classify request-body api_key as unplanted until D1 exists; check registry slots against the harness * test(security): classify request-body api_key under slot D1 * test(security): classify Langfuse and Datadog callback secrets under slots C1 and C3 * fix(cost-map): lower fireworks up-to-4b size tier to the pricing page price (#43740) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * fix(cost_calculator): bill chat per-second pricing once with a new cost_per_second field (#43614) * feat(cost_calculator): add cost_per_second for chat per-second pricing Keep legacy input_cost_per_second and output_cost_per_second as aliases for chat, completion, embedding and responses. When both legacy fields are set, input_cost_per_second wins Move Bedrock commitment rows to cost_per_second so they bill once Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(cost_calculator): drop legacy per-second fields from chat paths Keep Azure chat token pricing generic and update inert Voxtral rates and SageMaker examples Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost_calculator): recognize output-only per-second rates Include output_cost_per_second when checking whether a deployment cost entry has pricing so output-only legacy aliases remain attached to the deployment during cost selection Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(pricing): cover cost_per_second and legacy per-second aliases through the proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(cost_calculator): drop output_cost_per_second as a chat per-second alias Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(cost_calculator): restore output_cost_per_second as a chat per-second fallback Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost-map): keep input_cost_per_second on bedrock commitment rows for older clients Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(cost-map): add openai gpt-6-astra ultrafast tier prices from the pricing page (#43745) * chore(cost-map): add openai gpt-6-astra ultrafast tier prices from the pricing page Price-Sync: litellm-providers * feat(cost): support openai ultrafast tier fields in the model catalog --------- Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> Co-authored-by: Kerry <kerry@berri.ai> * fix(autorouter): compare historical and new savings consistently (#43348) * fix(autorouter): compare historical and new savings consistently * fix(autorouter): reject comparisons if request counts changed * fix(router): restore eligible LLM and classification breakdown * fix(router): avoid ambiguous baseline labels for partial comparisons * fix(model-prices): align Azure, Bedrock, Copilot, Gemini, Groq, OpenAI and OpenRouter entries with official docs (#43598) * fix(model-prices): correct azure/eu/gpt-6-astra to Data Zone rates Co-authored-by: rain <1504569896@qq.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model-prices): align groq, gemini and openai entries with official docs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model-prices): roll in verified Vertex, Gemini, OpenRouter and Azure AI registry fixes Absorbs the fields from #43609, #43666, #43671 and #43644 that match the provider's own docs or price API today, and adds a cost test for the azure/eu/gpt-6-astra Data Zone tiers Co-authored-by: bunnysayzz <stfuazzo@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model-prices): add Copilot, Bedrock Kimi K3, Gemini Robotics and OpenRouter values from official sources Co-authored-by: Michal Formanek <michal.formanek@generaliceska.cz> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: rain <1504569896@qq.com> Co-authored-by: bunnysayzz <stfuazzo@gmail.com> Co-authored-by: Michal Formanek <michal.formanek@generaliceska.cz> * fix(guardrails): enable explicit PANW MCP output scanning (#43109) * fix(guardrails): declare post_mcp_call for PANW Prisma AIRS Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): exercise post_mcp_call_hook dispatch in PANW tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model-catalog): add fal_ai resolution-tiered image cost fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): keep post_mcp_call opt-in for PANW Prisma AIRS Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model-catalog): add fal_ai resolution-tiered image cost keys to cost map schema Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: joshua <joshua@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> * refactor(rust): orchestrate Messages route execution (#43719) Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) * fix(streaming): keep the provider's served service_tier on streamed chunks and spend rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(streaming): satisfy type-discipline and strict ruff budgets Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(streaming): stamp the served service_tier on every Responses bridge chunk Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic-adapter): expose streamed chunks so disconnects bill partial spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(service-tier): cover anthropic and responses served-tier billing paths Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic-adapter): return a chunks-exposing stream so disconnects bill partial spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(service-tier): bill disconnects through the router's anthropic stream wrapper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style: apply ruff format to the anthropic stream changes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(coverage): ignore delegating properties the ast scan cannot see Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style: keep the cast-ok reasons on the cast call line Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover served service_tier billing for streamed chat and messages, complete and disconnected Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic-cache): delegate chunks/messages/model through the messages stream cache writer Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(streaming): keep service_tier on OpenAI-compatible parsed chunks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(streaming): parameterize delegated chunks and messages types Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(tests): follow the anthropic pass_through rename after merging main Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(anthropic): drain the logging worker between response cache tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend): cover azure, databricks, responses bridge and gemini served tiers in the stream billing integration test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(databricks): keep the served service_tier on streamed chunks and bill it Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(databricks): type the served service_tier chunk without a loose kwargs dict Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost): bill the served service_tier over the requested one Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(cost): drop explanatory comment from the tier resolution Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: kerry <kerry@berri.ai> * feat(bedrock): add openai gpt-6.1-sol global and base rows (#43758) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * perf(router): fetch cooldown state and usage counters in one Redis round trip (#43320) * perf(router): fetch cooldown state and usage counters in one Redis round trip The cooldown filter (CooldownCache) and usage-based-routing-v2 selection (LowestTPMLoggingHandler_v2) each issued their own MGET on every request because they live in different objects. RoutingReadBatch fetches both key sets through DualCache.async_batch_get_cache_shared while the healthy deployments are resolved and hands the usage slice to the strategy, so selection does not read again. Each cache keeps its own memory tier, throttling, reservation rollback and circuit-breaker handling, and the strategy falls back to its own read when the prefetch does not cover its keys. simple-shuffle keeps reading only cooldowns. aresponses no longer issues a second, blocking response-cache read from the worker thread that runs the sync wrapper. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(caching): keep per-cache tier failures inside the shared batch read Wrap the memory-tier prepare and backfill steps of DualCache.async_batch_get_cache_shared so a failing tier degrades that cache's read to None the way async_batch_get_cache does, instead of escaping into routing. Drop the aresponses sync-cache guard: for native Responses models the worker-thread read is the one whose key matches the write, so skipping it broke cached /v1/responses replays. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(router): rename usage key builder so the async cache-call check reads it as a key helper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(alerting): narrow daily-report cache values before numeric comparison Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(caching): type the shared batch-read helpers and merge Redis results without mutation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style: fix import sort in test_dual_cache Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(caching): flatten shared batch read keys without a stacked comprehension Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * perf(proxy): hold one spend counter batch across admission and across post-call accounting (#43369) Auth's spend counter MGET scope spans common checks, model budget check and reservation; reservation increments go out as one pipeline; post-call reconcile adjustments ride the ordinary increment pipeline and update_cache uses one batched read. Over-budget reservation counters are charged one at a time so a rejection never touches the counters after it; post-call counter keys are derived from ids without validating a UserAPIKeyAuth. Resolves LIT-8881 Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * perf(responses): run aresponses through the async wrapper so the cache is read once (#43769) * perf(responses): run aresponses through the async wrapper so the cache is read once aresponses() sets kwargs["aresponses"] = True and runs the decorated sync responses() on an executor, but _is_async_request() did not recognise that flag, so the sync wrapper did a second cache lookup on the executor thread with a differently ordered cache-key input. Every /v1/responses request paid two cache GETs against two different keys. Recognising aresponses in _is_async_request() leaves the async wrapper as the only cache reader and writer for the async path, one GET per request, same key on read and write Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(caching): let a responses cache entry cover aresponses so responses-only configs keep caching Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): bind Claude Code background sessions to their auto-router (#43767) Claude Code background sessions (claude --bg) stamp x-app: cli-bg on every request, including main-loop turns. The session router binding only accepted x-app: cli, so a background session never bound and its subagents' concrete-model calls bypassed the router. The binding write already requires the requested model to resolve to a pre-routing strategy, so background side calls naming plain models still never bind. Since f6eff1bde0 removed the clear path, the x-app check guarded nothing else. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(bedrock): add openai.gpt-6.1-sol us geo cris and Mantle rows (#43763) Co-authored-by: kerry-berri <kerry@berri.ai> * fix(proxy): recover session key owners from daily spend for usage attribution (#43642) * fix(proxy): recover daily spend key owners Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): simplify daily spend owner recovery Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(proxy): format daily activity metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover recovered owner metadata merge Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): bound the daily spend owner lookup with the statement timeout * test(integration): audit the daily activity key owner fallback on every usage route Thirty five integration cells under tests/integration/spend cover the daily spend owner fallback on all nine daily activity routes and /usage/ai/chat: the happy path per route, the unanimity rules (two users, blank and null rows, an owner the user table lacks, live and deleted keys with and without their own user, a spend log alias), a non admin reader, an invalid key, a 5 KB key, a locked LiteLLM_DailyUserSpend, 300 keys of one team, repeated reads, a second user landing between reads, a concurrent burst across the unified endpoints, a killed worker, and a proxy restart The traffic cells ignore the GET /v1/models call the proxy's five minute token limit refresh makes to every registered OpenAI compatible deployment, since it lands on a test's provider wire whenever the refresh instant falls inside the test --------- Co-authored-by: jesus <jesus@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * chore(cost-map): take azure context limits from models-sold-directly (#43759) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * perf(proxy): one request-scoped Redis pipeline for auth, spend, rate-limit and routing reads (#43407) RedisBatch: one pipeline per Redis backend for independently declared operations (MGET, GET, Lua scripts, INCRBYFLOAT, SET, DEL), a future per operation so each owner keeps its own fallback, Redis Cluster hash-slot fallback. A request-scoped batch middleware shares that pipeline across the auth identity reads and write-back, the spend counter MGET, the rate limiter Lua groups and the routing read. A rate-limit denial stands when another pipelined group fails; every pipelined group is refunded on rejection; local cooldowns win over the prefetch. The routing prefetch failure log line strips request line breaks (CodeQL py/log-injection) Resolves LIT-8882 Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): scope OpenAPI listings to the exact server prefix and drop upstream OAuth metadata when a server is saved (#43608) * fix(mcp): key discovery caches per caller correctly and drop stale caches on server updates Discovery-list cache identity now uses the hashed token instead of the raw api_key and treats MCPJWTSigner-signed servers as per caller. Server definition changes also drop the cached upstream OAuth metadata. OpenAPI listings look tools up under the normalized registry prefix with the separator, so an overlapping sibling prefix no longer leaks into the list. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): keep the discovery cache digest call unchanged so CodeQL matches the existing alert Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): guard OAuth metadata cache writes with a per-server generation and drop unproven per-caller discovery keys An upstream metadata fetch that started before a server edit could store its stale reply after invalidate_oauth_metadata_cache ran. Invalidation now bumps a per-server generation and the fetch only stores when the generation it captured before I/O is unchanged. The MCPJWTSigner-based per-caller discovery classification and the api_key to token key change had no reproduction (the signer only injects on tools/list, and UserAPIKeyAuth hashes api_key in place), so both go back to the merge-base behavior. Integration coverage under tests/integration/mcp: overlapping OpenAPI aliases, a config-declared server name with a space, OAuth metadata refetch after a save, and the in-flight stale-write race Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): keep OAuth metadata generations only while a fetch is in flight Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): count queued OAuth metadata fetchers so invalidation survives lock handoff Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): keep a held OAuth metadata lock registered even when no fetcher slot claims it Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): prove a peer worker drops stale upstream OAuth metadata after a save elsewhere Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * perf(proxy): one post-call Redis pipeline per backend for spend, rate-limit, routing and response-cache writes (#43779) Post-call owners declare into one request-scoped RedisBatch per Redis backend: spend counter increments and reservation reconciliation, rate-limit token Lua updates and refunds, parallel-slot release (freed locally at once), deployment TPM, and compatible async response-cache SETs. The batch is sent once the success and failure callbacks have run, or on a deadline, and pending batches are drained at shutdown before Redis disconnects. nx writes, non-Redis caches and calls outside a request stay direct; numeric string TTLs keep the direct-path coercion. Resolves LIT-8883 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: yassin <yassin@berri.ai> * fix(bedrock): set gpt-6.1-sol max output tokens to 131072 (#43782) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * perf(proxy): refresh auth management objects through the request Redis pipeline (#43776) Identity objects (key, end user) load through the request MGET and their write-backs, the registry reads and the management-object SETs ride the request pipeline. A team refresh invalidates its alias with a pipelined DEL instead of a synchronous DEL plus a duplicate async one, and an MGET miss is remembered so no per-key GET follows it in the same request. Resolves LIT-9012 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: yassin <yassin@berri.ai> * feat(agents): add identity storage and validation contracts (#43720) * feat(agents): identity storage and contracts * fix(agents): cache positive identity lookups with fresh policy checks * test(agents): include identity attribution in spend fixture --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> * bump: litellm-enterprise 0.1.71 -> 0.1.72, litellm-proxy-extras 0.4.102 -> 0.4.103, litellm 1.104.0 -> 1.105.0 (#43789) * fix(auth): give UI/CLI session tokens their own AES-GCM context and header-safe shape (#43790) * refactor(auth): bind UI/CLI session tokens to their own AES-GCM context UI and CLI session tokens are now always encrypted with AES-256-GCM and a fixed session associated-data value, and the session-token check only accepts AES-GCM values carrying that same value. Stored secrets keep their current encryption and decrypt unchanged, so nothing needs migrating. encrypt_value_helper and decrypt_value_helper take an optional aad. XSalsa20 cannot bind associated data, so an AAD-bound value is always written as AES-256-GCM, and an AAD-bound decrypt refuses the legacy format. Session tokens issued before the upgrade stop validating, so UI and CLI users sign in once more after upgrading. * test(e2e): cover real SSO login through the dashboard and the lite CLI Adds two specs under tests/e2e/ui/oidc, run by playwright.oidc.config.ts against a live Keycloak stack. The dashboard spec checks that the SSO session authorizes the Virtual Keys and Models data requests. The CLI spec runs a real lite login in an isolated HOME with the keyring disabled, then lists models and sends one chat completion with the stored session. The main Playwright config now ignores oidc/. * fix(auth): encode UI/CLI session tokens as unpadded base64url Session tokens carried the v2:gcm: storage prefix and base64 padding. Basic-auth parsers split on the first colon and browsers reject ':' and '=' in WebSocket subprotocols, so Langfuse pass-through and the realtime playground could not use them Tokens are now plain unpadded base64url, the same header-safe shape as any bearer token * fix(auth): prefix UI/CLI session tokens with litellm_login_ A prefix-less token starts with sk- about once in 262,144 logins and is then routed as a virtual key, so that login gets a 401. The prefix also makes session tokens easy to spot in logs The prefix doubles as the token's AES-GCM associated data, so the visible kind and the encrypted kind cannot disagree --------- Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> * chore(deps): bump pyjwt, moment and brace-expansion to clear osv-scan (#43792) pyjwt 2.13.0 -> 2.14.0 (uv.lock only, pyproject floor unchanged), moment 2.30.1 -> 2.31.0 and the brace-expansion override 5.0.9 -> 5.0.12 in the dashboard. oauthlib's only fixed release (4.0.0, 2026-09-28) is still inside the 3-day uv exclude-newer cooldown, so its two findings are ignored until 2026-10-02 * fix(ui): surface x-litellm-call-id in Logs search, table and drawer (#42436) * fix(ui): surface x-litellm-call-id in Logs search, table and drawer Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(ui): regenerate api types for spend logs search description Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(e2e): drop redundant comments from the call id logs helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(e2e): format logs call id helper and spec Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): keep one id per Logs row, move x-litellm-call-id to hover and drawer The Request ID cell shows only request_id again. When the row's litellm_call_id differs, the cell tooltip lists it as x-litellm-call-id with its own copy button, and the drawer header labels the second line x-litellm-call-id: instead of the call id caption. Stacking two ids in every row made the column noisy for the common case where the viewer only needs the row they searched for. * test(e2e): cover the Request ID tooltip hover and copy path Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: poll the clipboard after the tooltip copy and drop a jsdom aside The e2e read navigator.clipboard right after the click, so a slow async write could fail the check even though copy works. The unit test's fireEvent choice (jsdom has no layout, so a real pointer move off the trigger closes the tooltip before the click lands) is documented here instead of inline. --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> * fix(proxy): look up hashed key names with two spend log rows per key (#43656) * fix(proxy): look up hashed key names with two spend log rows per key The spend-log fallback for keys missing from the key table read every row per key to check that all named rows agreed, which passed the 5s statement timeout on busy keys even with the (api_key, startTime) index. Probe only the oldest and newest named row per key, so the lookup stays two index reads per key however much the key logged. * fix(proxy): cap each spend log name probe at 100 rows per key * fix(proxy): bound the newest-row probe at where the oldest probe stopped The newest-row probe now starts at the row where the oldest-row probe gave up, so a key with under 200 rows in the window is read once instead of twice, and the lookup transaction turns bitmap scans off so the planner walks the (api_key, startTime) index instead of every row of a busy key when statistics or the visibility map are stale. * test(integration): add spend log alias probe cells for the daily activity routes --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * fix(cost-map): add deprecation_date to two together_ai nvidia rows (#43809) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * test(ci): refresh retired OpenAI tool-call models (#43676) * fix(ui): keep MCP permissions visible after key, team and MCP server saves (#43810) * fix(ui): keep MCP permissions visible after key, team and MCP server saves Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): type the object_permission include as a prisma TypedDict Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): do not block key save confirmation on cache refetch Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ci): repair MCP Responses and budget fixtures (#43788) * test(ci): repair MCP Responses and budget fixtures * test(auth): verify delegated budget changes persist * test(bedrock): accept regional aliases that inherit Converse routing (#43785) * test(bedrock): accept regional aliases that inherit Converse routing * test(bedrock): cover regional alias metadata independently of catalog * fix(params): filter internal traceback flag from provider requests (#43783) * perf(router): honour the cooldown read interval in the routing prefetch (#43815) Resolves LIT-9043 Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): carry per-request routing reads on context variables instead of public method kwargs (#43814) Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(bedrock): add beta header for output config in message (#43778) * refactor: clean up fresh tech debt from 2026-09-29 (#43830) * refactor: clean up fresh tech debt from 2026-09-29 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(routing): pin usage-based routing Redis reads through the proxy and SDK Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(router): settle the shared logging worker before recording shadow callbacks (#43847) * test(router): settle the shared logging worker before recording shadow callbacks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(router): always stop the shared logging worker after settling it Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(model_prices): add Gemini Veo, Mistral and Azure Claude 4.5 deprecation dates (#43857) * fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates (#43764) * refactor(rust): centralize bridge execution wrappers (#43871) Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(cost-map): add openai gpt-image-2.5 batch prices from the pricing page (#43869) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * chore(cost-map): add fireworks priority prices for ember-1, nemotron and glm 5.3 us rows (#43811) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * fix(proxy): attribute completed batch cost rows to /batches in daily activity (#43870) * fix(proxy): attribute completed batch cost rows to /batches in daily activity Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend): wait for priced batch tokens before asserting team endpoint activity Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): strip encrypted reasoning the pinned deployment cannot decrypt (#43781) Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost_calculator): stop copying optional_params into response hidden params (#43637) * fix(cost_calculator): stop copying optional_params into response hidden params * test(cost_calculator): assert the stored spend-log request and logging payload carry no forwarded credentials * feat(pricing): add vertex_ai gemini-3.8 flash tts rows (#43876) Price-Sync: litellm-providers Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> * fix(proxy): keep request-body credentials out of stored spend-log requests (#43635) * fix(proxy): keep request-body aws credentials out of stored spend-log requests * fix(proxy): redact every credential-named request-body field in stored spend-log requests Replace the hard-coded AWS key check in the spend-log request-body sanitizer with SensitiveDataMasker's key classification, so Azure, Vertex, watsonx, OCI, GigaChat, Gemini and header credentials are redacted too. Proxy-stamped key identity metadata is kept. * fix(proxy): keep request identifiers named like keys in stored spend-log requests * refactor(proxy): drop the AWS-only snapshot exclusion now that spend-log redaction is name-based * refactor(proxy): use SensitiveDataMasker's key classification without an exclusion list * refactor(proxy): always redact credential-named fields in stored spend-log payloads * fix(ui): right-align money and count columns across tables (#37889) * fix(ui): right-align money and count columns across tables Make numeric the one alignment token for the three shared table wrappers (DataTable, MemberTable, SimpleTable) via NUMERIC_CELL_CLASS, and flag every money, cost, and bare-count column that was still left-aligned. Raw ui/table usages that render spend or budgets get the same class on their header and cell. * test(ui): render the organizations alignment test with providers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): query alignment cells by role instead of DOM traversal Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(agents): enforce authoritative agent permissions (#43721) * feat(agents): authoritative permissions * fix: enforce authoritative managed agent permissions * fix(agents): only consult the identity store for managed targets is_agent_allowed entered the identity-store path whenever a prisma client was configured, so an ordinary agent paired with an internal user returned 503 instead of 200. Classify the target from the registry first and fall back to the store only when the registry has no entry, so an unmanaged target never depends on the store being reachable. * fix(agents): gate the managed path on an admitted policy object Ten call sites branched on `managed_agent_policy is not None`, which any MagicMock attribute satisfies, so the managed path fired on unmanaged subjects and died in Pydantic validation as a 503. Route every check through a shared helper that requires a real AgentResponse. * test(mcp): stub the writer replica the fresh-policy reads use reload_admitted_user now passes check_db_only through to get_user_object, so the user row is read from writer_db. Point the mocks at the replica the code actually reads and give each parametrized case its own user id. * fix(agents): cap a managed agent at the invoking team's agents resolve_agent_access returned the managed policy's grants before the agent_caller ceiling was applied, so a managed agent acting on behalf of a user reached agents that user's team was never granted. Intersect with the caller ceiling the unmanaged path already honours. * fix(agents): restore token narrowing and scope the private-access suppressions The managed-model check lost its valid_token narrowing when it moved to the shared helper. Make the caller-access resolver public rather than reaching into it from module scope, and give each remaining private access a reason. * docs(agents): drop the comment claiming admins skip the A2A permission check The check has never had an admin bypass on this path, so the comment described behaviour the code does not implement. * test(proxy): stub the writer reads and restore the MCP manager singleton Fresh-policy user lookups read writer_db, so the team and rest-endpoint mocks stubbed a replica the code no longer reads, and the dashboard session fake still had the pre-kwarg signature. The manager reload also rebound global_mcp_server_manager in every MCP module without restoring it, leaking an empty manager into later files. * style: sort imports under the litellm package ruff config * fix(mcp): cap a managed agent's servers and tools at the invoking caller managed_agent_servers and managed_agent_tools returned the agent's own grants without the agent_caller ceiling the unmanaged resolvers apply, so a managed agent reached MCP servers and tools the echoed caller could not. Call the existing ceiling helpers on both axes. * refactor(mcp): return the caller-capped tools without an interim list The ceiling helper already returns a sequence, so materializing it into a list added a mutable collection for nothing. Sort at the return sites instead, which also makes the tool order stable across both branches. * fix(agents): preserve actor ceilings during managed target checks * fix(agents): keep managed permission ceilings authoritative * fix(mcp): fail closed on authoritative caller team outages --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> * fix(proxy): strip caller credentials from websocket passthrough (#43855) * fix(proxy): strip caller credentials from websocket passthrough Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover configured x-api-key in websocket passthrough credential test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: oliver <oliver@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): render access group MCP and agent selections as wrapping chips (#41228) * fix(ui): render access group MCP and agent selections as wrapping chips Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): cover 20 selected MCP servers rendering as separate chips Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): cover MCP and agent chip selection in access group create dialog Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> * fix(responses): scan and mask top-level instructions with guardrails (#43629) * fix(responses): scan and mask top-level instructions with guardrails The Responses guardrail translation handler put a non-empty top-level instructions field into structured_messages as a system row but never into the flat texts list, so guardrails that scan texts skipped it, flat-text masking could not rewrite it, and PANW latest-only selection failed its alignment guard whenever instructions were present. Seed texts with the instructions row, carry that offset into the flat-text write-back so a rewritten row lands on data["instructions"], and account for the leading row in the PANW Responses alignment. Resolves LIT-8931 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses): reject empty guardrail rewrites instead of forwarding raw input An explicit texts=[] answer from a guardrail now fails the count check and raises UnappliableRequestRewrite like any other misaligned rewrite; only a missing texts key means no rewrite. Types the out-param as dict[str, object] and adds integration coverage for instructions blocking, masking, empty instructions, tool loops, latest-only and concurrent workers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(responses): type the texts-replacing guardrail helper explicitly Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses): honor skip_system_message_in_guardrail for instructions and system input items Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(responses): cover skip_system_message_in_guardrail on the live proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses): keep skipped rows through full-coverage rewrites and align latest-only with skip_system Trust a guardrail's structured_messages_cover_full_request claim only when it returns as many rows as the full normalized request, otherwise merge the scoped rows back so skipped instructions and system items survive the write-back. Make PANW's Responses reasoning alignment skip-aware so latest-only still picks the latest user turn when system content is excluded from texts. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(responses): annotate new guardrail tests with return types Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses): treat an empty guardrail texts answer as no rewrite like chat completions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(responses): type the guardrail test doubles explicitly Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(traces): add Rust storage foundation (#43819) * wip * feat(traces): establish shared Rust storage foundation * fix(traces): escape ClickHouse text parameters * test(traces): exercise response cap with bounded strings * fix(traces): remove unnecessary lint expectation * fix(traces): encode ClickHouse timestamp units in Rust * test(traces): mark exception match as a regex * refactor(traces): execute schema setup in Rust * refactor(traces): use shared logging execution wrapper * docs(traces): replace foundation README with boundary rules * fix(traces): use current bridge execution facade * fix(traces): account for protocol cast in lint budget Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+…

devin-ai-integration Bot added 2 commits September 24, 2026 03:27

fix(streaming): keep the provider's served service_tier on streamed c…

0b1ccf8

…hunks and spend rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

fix(streaming): satisfy type-discipline and strict ruff budgets

e6722a9

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot requested a review from a team September 24, 2026 03:42

devin-ai-integration Bot had a problem deploying to e2e-changed September 24, 2026 03:42 Error

greptile-apps Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread litellm/completion_extras/litellm_responses_transformation/transformation.py

cursor Bot reviewed Sep 24, 2026

View reviewed changes

fix(streaming): stamp the served service_tier on every Responses brid…

69fe084

…ge chunk Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot had a problem deploying to e2e-changed September 24, 2026 04:09 Error

cursor Bot reviewed Sep 24, 2026

View reviewed changes

mateo-berri requested a review from litellm-stampbot September 24, 2026 04:35

litellm-stampbot and others added 2 commits September 25, 2026 00:43

fix(anthropic-adapter): expose streamed chunks so disconnects bill pa…

7039b6d

…rtial spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

test(service-tier): cover anthropic and responses served-tier billing…

909bb4c

… paths Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

greptile-apps Bot reviewed Sep 25, 2026

View reviewed changes

Comment thread litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py Outdated

cursor Bot reviewed Sep 25, 2026

View reviewed changes

Comment thread litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py Outdated

litellm-stampbot and others added 3 commits September 25, 2026 01:28

fix(anthropic-adapter): return a chunks-exposing stream so disconnect…

bf86914

…s bill partial spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

test(service-tier): bill disconnects through the router's anthropic s…

047b506

…tream wrapper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into litellm_stream_served…

38ee5f1

…_service_tier Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> # Conflicts: # tests/test_litellm/litellm_core_utils/test_litellm_logging.py

cursor Bot reviewed Sep 27, 2026

View reviewed changes

greptile-apps Bot reviewed Sep 27, 2026

View reviewed changes

Comment thread litellm/llms/databricks/chat/transformation.py Outdated

fix(databricks): type the served service_tier chunk without a loose k…

bc9cb9f

…wargs dict Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot had a problem deploying to e2e-changed September 27, 2026 01:55 Error

fix(cost): bill the served service_tier over the requested one

4db1654

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot had a problem deploying to e2e-changed September 29, 2026 17:11 Error

refactor(cost): drop explanatory comment from the tier resolution

aaf6c94

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot requested a deployment to e2e-changed September 29, 2026 17:12 Waiting

cursor Bot reviewed Sep 29, 2026

View reviewed changes

mateo-berri approved these changes Sep 29, 2026

View reviewed changes

litellm-stampbot merged commit e814532 into main Sep 29, 2026
104 of 108 checks passed

litellm-stampbot deleted the litellm_stream_served_service_tier branch September 29, 2026 19:54

yuneng-berri mentioned this pull request Oct 1, 2026

test(ci): repair stale tests and flaky CI infrastructure #43983

Merged

1 of 23 tasks

devin-ai-integration Bot mentioned this pull request Oct 13, 2026

fix(databricks): accept service_tier on requests and carry the served tier on responses #46111

Merged

4 of 5 tasks

kerry-berri mentioned this pull request Oct 13, 2026

fix(databricks): carry service_tier through request, streaming and cost calc #41837

Closed

5 of 6 tasks

Conversation

devin-ai-integration Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Integration coverage across the related tickets

Linear ticket

Pre-Submission checklist

Screenshots / Proof of Fix

Before (a3d791f)

Streamed chunks carry the served tier

Spend row records the served tier

After (e6722a9)

Streamed chunks carry the served tier

Spend row records the served tier

Responses bridge (gpt-5-nano, deployment nano5-tier), same curl

/v1/messages disconnect (gpt-4.1-nano, chat-adapter route)

Integration matrix (scripted upstream, real proxy, Postgres and Redis cache on)

Endpoint x outcome coverage

Type

Caveats (if any)

Low

QA runbook

Final Attestation

Uh oh!

devin-ai-integration Bot commented Sep 24, 2026

Uh oh!

mateo-berri commented Sep 24, 2026

Uh oh!

mateo-berri commented Sep 24, 2026

Uh oh!

greptile-apps Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codspeed Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

codecov Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

CLAassistant commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mateo-berri commented Sep 24, 2026

Uh oh!

mateo-berri commented Sep 24, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

litellm-stampbot commented Sep 25, 2026

Uh oh!

litellm-stampbot commented Sep 25, 2026

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

litellm-stampbot commented Sep 27, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

litellm-stampbot commented Sep 27, 2026

Uh oh!

litellm-stampbot commented Sep 27, 2026

devin-ai-integration Bot commented Sep 24, 2026 •

edited

Loading

Before (`a3d791f`)

After (`e6722a9`)

Responses bridge (gpt-5-nano, deployment `nano5-tier`), same curl

greptile-apps Bot commented Sep 24, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

codspeed Bot commented Sep 24, 2026 •

edited

Loading

codecov Bot commented Sep 24, 2026 •

edited

Loading

CLAassistant commented Sep 24, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading